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DETAILED ACTION 

Information Disclosure Statement 

1 . The information disclosure statement (IDS), filed on 01/1 7/2002, has been placed in the 
application file, and the information referred to therein has been considered as to the merits. 

Claim Objections 

2. Claim 13 is objected to because of the following informalities: In claim 13, the limitation 
beginning with lettering "(c)" should be "(b)". Appropriate correction is required. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

4. Claims 1-5, 8, 10, 13-16, 18- 20, and 22 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Carlson (U.S. Pat. #4,554,585). 

For claim 1, Carlson teaches an electronic imaging system (fig. 1, 100, 1 10, 108) for 
capturing an image of a scene (col. 2, lines 57-62), said imaging system comprising: 

(a) an optical system (fig. 1, 100) for producing an optical image of the scene (col. 2, line 
63 -col. 3, line 10); 

(b) an imaging sensor (solid-state imager, col. 2, lines 63-65) having a surface in optical 
communication (col. 2, line 66 - col. 3, line 2) with the optical system; and 
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(c) a plurality of imaging elements (fig. 2a) distributed on the surface of the imaging 
sensor (col. 4, lines 13-23) for converting the optical image into a corresponding output signal 
(col. 3, lines 4-7), said imaging elements located according to a distribution representable by a 
nonlinear function in which the relative density of the distributed imaging elements is greater 
toward the center of the sensor (col 4, lines 28-33), wherein the distribution provides physical 
coordinates for each of the imaging elements corresponding to a projection of the scene onto a 
non-planar surface, thereby compensating for perspective distortion of the scene onto the non- 
planar surface (col. 4, lines 24-28). 

For claim 2, Carlson further discloses a system wherein the non-planar surface is 
inherently a cylinder. This is inherent because in col. 4, lines 28-33, Carlson states that the 
discrete picture elements 200 are symmetrically disposed about the center of a polar-coordinate 
spatial distribution pattem. 

For claim 3, Carlson further discloses a system wherein the non-planar surface is a 
sphere. This is inherent because in col. 4, lines 28-33, Carlson states that the discrete picture 
elements 200 are symmetrically disposed about the center of a polar-coordinate spatial 
distribution pattem. 

. For claim 4, Carlson further discloses an optical system that includes a lens (fig. 3, 304) 
and the axis of rotation of the cylinder intersects a nodal point of the lens. As stated before, it is 
inherent that Carlson's non-planar surface is a cylinder because in col. 4, lines 28-33, Carlson 
states that the discrete picture elements 200 are symmetrically disposed about the center of a 
polar-coordinate spatial distribution pattem. As illustrated in fig. 3, the center of the cylinder is 
located at a nodal point of the lens because the imager is located along the optical axis of the lens 
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(col. 5, lines 13-19). The imager senses light from the lens via a low-pass filter (col. 5, lines 26- 
29). 

For claim 5, Carlson further discloses a system wherein the optical system includes a lens 
and the center of the sphere is located at a nodal point of the lens. As stated before, it is inherent 
that Carlson's non-planar surface is a sphere because in col. 4, lines 28-33, Carlson states that the 
discrete picture elements 200 are symmetrically disposed about the center of a polar-coordinate 
spatial distribution pattern. As illustrated in fig. 3, the center of the sphere is located at a nodal 
point of the lens because the imager is located along the optical axis of the lens (col. 5, lines 13- 
19). The imager senses light from the lens via a low-pass filter (col. 5, lines 26-29). 

For claim 8, Carlson further discloses a system wherein the imaging sensor is a charge- 
coupled device (col. 2, lines 63-65). 

For claim 10, Carlson further discloses a system wherein the output signal includes data 
from a plurality of images (col. 2, lines 19-33). 

For claim 13, Carlson teaches an electronic image sensor (solid-state imager, col. 2, lines 
63-65) for use in an electronic imaging system (fig. 1, 100, 1 10, 108) for capturing an optical 
image of a scene (col. 2, lines 57-62); said sensor comprising: 

(a) a surface in optical commimication with the scene (col. 2, line 63 - col. 3, line 10); 

and 

(c) a plurality of imaging elements (fig. 2a) distributed on the surface of the sensor (col, 
4, lines 13-23) for converting the optical image into a corresponding output signal (col. 3, lines 
4-7), said imaging elements located on the surface according to a distribution representable by a 
nonlinear function in which the relative density of the distributed imaging elements is greater 



Application/Control Number: 1 0/052,020 Page 5 

Art Unit: 2612 

toward the center of the sensor (col. 4, lines 28-33), wherein the distribution provides physical 
coordinates for each of the imaging elements corresponding to a projection of the scene onto a 
non-planar surface, thereby compensating for perspective distortion of the scene onto the non- 
planar surface (col. 4, lines 24-28). 

Regarding claim 14, this claim is a claim corresponding to the claim 2. Therefore, claim 

14 is analyzed and rejected as previously discussed with respect to claim 2. 

Regarding claim 15, this claim is a claim corresponding to the claim 3. Therefore, claim 

15 is analyzed and rejected as previously discussed with respect to claim 3. 

Regarding claim 16, this claim is a claim corresponding to the claim 8. Therefore, claim 

16 is analyzed and rejected as previously discussed with respect to claim 8. 

For claim 18, Carlson discloses a digital camera (fig. 1, 100, 1 10, 108) containing the 
image sensor (solid-state imager, col. 2, lines 63-65) of claim 13. In col. 3 line 47, the image 
signal processor of Carlson's camera digital. 

For claim 19, Carlson discloses in an electronic imaging system (fig. 1, 100, 110, 108) 
including an image sensor (solid-state imager, col. 2, lines 63-65) for converting an optical 
image into an output signal (col. 3, lines 4-7), the improvement wherein the image sensor 
comprises a plurality of imaging elements (fig. 2a) distributed on a surface of the sensor (col. 3, 
lines 4-7) for converting the optical image into a corresponding output signal (col. 3, lines 4-7); 
said imaging elements located on the surface according to a distribution representable by a 
nonlinear function in which the relative density of the distributed imaging elements is greater 
toward the center of the sensor (col. 4, lines 28-33), thereby compensating for perspective 
distortion of the scene onto a non-planar surface (col. 4, lines 24-28). 
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Regarding claim 20, this claim is a claim corresponding to the claim 8. Therefore, claim 
20 is analyzed and rejected as previously discussed with respect to claim 8. 

For claim 22, Carlson inherently discloses a system including the improvement wherein 
the image sensor (solid-state imager, col 2, lines 63-65) is a component in a digital camera (fig. 
1, refs. 100, 110, 108). In col. 3, line 47, Carlson teaches that the image signal processor of the 
camera is digital. 

Claim Rejections - 35 USC § 103 

5. This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7. Claims 6-7 are rejected under 35 U.S.C. 103(a) as being unpatentable over Carlson (U.S. 
Pat. #4,554,585) in view of Hsieh et al. (U.S. Pat #6,798,923). 
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For claim 6, Carlson does not specifically disclose a system wherein the radius of the 
cylinder is a function of a focal length of the optical system. However, in col. 4 lines 28-33, 
Carlson states that the discrete picture elements 200 are symmetrically disposed about the center 
of a polar-coordinate spatial distribution pattern. Additionally, in figure 2a, Carlson illustrates an 
image sensor where there are radial changes with respect to the high/low resolution periphery 
(col. 4, lines 24-42). The center of the cylindrical imager, which senses light from the lens via a 
low-pass filter (col. 5, lines 26-29), is located along the optical axis of the lens as illustrated in 
fig. 3 (col. 5, lines 13-19). In the same field of endeavor, Hsieh explains how to remove the 
effects of panoramic distortion of images projected on a cylinder (col. 3, line 65 - col. 4, line 40) 
by utilizing equations (1) - (4) to obtain coordinates for the corresponding pixel of the image 
plane in the cylindrical map. Since Hsieh states that the radius is equal to the focal length, the 
focal length will change with respect to the radius. Therefore, it would have been obvious to one 
of ordinary skill in the art at the time the invention was made for Carlson to implement a system 
wherein the radius of the cylinder is a function of a focal length of the optical system in order to 
permit a predetermined mapping of the image onto the sensor. 

For claim 7, Carlson does not specifically disclose a system wherein the radius of the 
sphere is a function of a focal length of the optical system. However, in col. 4 lines 28-33, 
Carlson states that the discrete picture elements 200 are symmetrically disposed about the center 
of a polar-coordinate spatial distribution pattern. Additionally, in figure 2a, Carlson illustrates an 
image sensor where the radius changes with respect to the high/low resolution periphery (col. 4, 
lines 24-42). The center of the spherical imager, which senses light from the lens via a low-pass 
filter (col. 5, lines 26-29), is located along the optical axis of the lens as illustrated in fig. 3 (col. 
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5, lines 13-19). In the same field of endeavor, Hsieh explains how to remove the effects of 
panoramic distortion of images projected on a sphere (col. 3, line 65 - col. 4, line 40). Although, 
Hsieh gives an example of the cylindrical geometry, Carlson utilizes polar coordinates as 
explained in claim 3. Since Hsieh states that the radius is equal to the focal length, the focal 
length will change with respect to the radius. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made for Carlson to implement a system 
wherein the radius of the sphere is a function of a focal length of the optical system in order to 
permit a predetermined mapping of the image onto the sensor. 

8. Claims 9, 17, and 21 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Carlson (U.S. Pat. #4,554,585) 

For claim 9, Carlson does not specifically disclose a system wherein the imaging sensor 
is a CMOS device. However, in col. 2, lines 63-64, Carlson teaches that his imaging system 
includes a solid-state imager. Examiner takes Official Notice that it is well known in the art for a 
CMOS device to be a type of solid-state imager. It would have been obvious to one of ordinary 
skill in the art at the time the invention was made for Carlson to provide a system wherein the 
imaging sensor is a CMOS device in order to provide low-power consumption and to allow 
processing circuits to be integrated on the same substrate. 

Regarding claims 17 and 21, both of these claims are claims corresponding to the claim 

9. Therefore, claims 17 and 21 are analyzed and rejected as previously discussed with respect to 
claims 9. 

9. Claims 11-12 are rejected imder 35 U.S.C. 103(a) as being unpatentable over Carlson 
(U.S. Pat, #4,554,585) in view of Ribera et al. (U.S. Pat. # 6,603,503). 
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For claim 11, Carlson discloses a system with a moveable television camera that 
produces a video signal, which is coupled to an image signal processor. Carlson's image signal 
processor, which analyzes the image defined by the video signal to determine the exact 
whereabouts of a particular object in field of view, can inherently operate directly on the output 
signal without having to warp the image data (col. 3, lines 33-46). However, Carlson does not 
teach a system including a processor for combining the images into a composite image. 

In the same field of endeavor, Ribera discloses a system including a processor (Ribera, 
fig. 4, ref 10) for combining the images into a composite image, thereby the processor can 
operate directly on the output signal without having to warp the image data (Ribera, col. 6, lines 
10-21 and col. 4, lines 51-55). Similar to Ribera, Carlson's invention is related to wide view 
images (Carlson, col. 2, line 67 - col. 3, line 1). Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made for Carlson to implement a system 
including a processor for combining the images into a composite image, thereby the processor 
can operate directly on the output signal without having to warp the image data in order to 
display panoramic images over a substantially 360 degree by 360 degree range of angles (Ribera, 
col. 1, lines 36-39). 

For claim 12, Carlson does not further disclose a system including a projector for 
projecting the composite image onto a planar surface. However, Ribera fiirther discloses a 
system including a projector (Ribera, fig. 4, ref 20) for projecting the composite image onto a 
planar surface (Ribera, col. 6, lines 10-21). Similar to Ribera, Carlson's invention is related to 
panoramic/wide view images (Carlson, col. 2, line 67 - col. 3, line 1). Therefore, it would have 
been obvious to one of ordinary skill in the art at the time the invention was made for Carlson to 
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implement a system including a projector for projecting the composite image onto a planar 
surface in order to display panoramic images over a substantially 360 degree by 360 degree 
range of angles (Ribera, col. 1, lines 36-39). 

10. Claims 23-27 are rejected under 35 U.S.C. 103(a) as being unpatentable over Carlson 
(U.S. Pat. #4,554,585) in view of Huang et al. ("Panoramic Stereo imaging System with 
Automatic Disparity Warping and Seaming," Graphical Models and image Processing, Vol. 60, 
No. 3, May 1998, pp. 196-208.) 

For claim 23, Carlson teaches a method (fig. 1, 100, 1 10, 108) comprising: (a) 
generating the source digital images from an imaging source having imaging elements 
distributed so as to compensate for perspective distortion of the scene onto a non-planar surface 
(col. 2, lines 63-65; col. 3, lines 4-7 and 47; col. 4, lines 28-33). This claim differs from Carlson 
in that he does not teach a method of generating a composite digital image from at least two 
source digital images, said method comprising: 

(b) combining the source digital images to form a composite digital image. 

In the same field of endeavor, Huang teaches a method of generating a composite digital 
image from at least two source digital images (page 197, section 3.1, paragraph 1), said method 
comprising: (b) combining the source digital images to form a composite digital image (page 
200, section 3.5). Additionally, Huang's panoramic stereo imaging system inherently has an 
imaging sensor because his system includes two cameras for the left-eye and the right-eye (page 
197, section 3.1, paragraph 2). This system generates focused images by selecting the correctly 
focused image for each sensor (pages 197-198, section 3.2, paragraphs 1-2). Please see figs. 5-6 
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and read pages 199-200, section 3.4, paragraphs 1-2. Similar to Huang, Carlson discloses an 
imaging system for image warping/ blurring improvements (Carlson, Abstract). It would have 
been obvious to one of ordinary skill in the art at the time the invention was made for Carlson to 
provide a method of generating a composite digital image from at least two source digital images 
in order to provide 360® panoramic stereo images (Huang, section 2, page 197, paragraphs 2-4), 

For claim 24, Carlson, as modified by Huang, further discloses a system including a 
projector for projecting the composite image onto a planar surface. On pages 204-207, Huang 
explains and illustrates the experimental results of the panoramic stereo imaging system. He 
demonstrates how the image distortion correction of the composite image is projected onto a 
planar surface. It would have been obvious to one of ordinary skill in the art at the time the 
invention was made for Carlson to provide a method of including a projector for projecting the 
composite image onto a planar surface in order to provide 360° panoramic stereo images (Huang, 
section 2, page 197, paragraphs 2-4). 

For claim 25, Carlson does not teach a method wherein the two source digital images 
overlap in overlapping pixel regions. Huang teaches a method wherein the two source digital 
images overlap in overlapping pixel regions (Huang, page 207, section 5 - Conclusion). It 
would have been obvious to one of ordinary skill in the art at the time the invention was made 
for Carlson to provide a method wherein the two source digital images overlap in overlapping 
pixel regions in order to provide 360° panoramic stereo images (Huang, section 2, page 197, 
paragraphs 2-4). 

For claim 26, Carlson, as modified by Huang, further discloses a method wherein the 
non-planar surface is a cylinder. In col. 4 lines 28-33, Carlson states that the discrete picture 
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elements 200 are symmetrically disposed about the center of a polar-coordinate spatial 
distribution pattern. It is well known in the art for coordinates of a cylinder to be known as polar- 
coordinates. Also, please read Huang, pages 199-200, section 3.4, paragraphs 1-2. 

For claim 27, Carlson, as modified by, Huang further discloses a system wherein the 
non-planar surface is a sphere. In col. 4 lines 28-33, Carlson states that the discrete picture 
elements 200 are symmetrically disposed about the center of a polar-coordinate spatial 
distribution pattern. It is well known in the art for coordinates of a sphere to be known as polar- 
coordinates. In the disclosure of Huang, 3-D cameras are utilized in the panoramic stereo 
imaging system. Although he uses a cylindrical surface to describe his panoramic stereo 
imaging system, Huang makes readers aware of the need for image distortion correction for 
images at the proper positions of a sphere as well as a cylinder. 

Conclusion 

1 1 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Carramah J. Quiett whose telephone number is (703) 305-0566. 
The examiner can normally be reached on 8:00-5:00 M-F. Beginning March 2005, the 
examiner's telephone number will be changed to (571) 272-7316. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Wendy Garber can be reached on (703) 305-4929. Beginning March 22. 2005. the 
supervisor's telephone number will be changed to (571) 272-7308. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



C.J.Q. 

March 7, 2005 




